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Abstract 

We prove n^+^^(^/p) /p'^'^'^^ lower bounds for the space complexity of p-pass streaming algorithms 
solving the following problems on n- vertex graphs: 

• testing if an undirected graph has a perfect matching (this implies lower bounds for computing a 
maximum matching or even just the maximum matching size), 

• testing if two specific vertices are at distance at most 2{p + 1) in an undirected graph, 

• testing if there is a directed path from s to i for two specific vertices s and < in a directed graph. 

Before our result, it was known that these problems require fi(n^) space in one pass, but no ri^+^^'^' 
lower bound was known for any p > 2. 

These streaming results follow from a communication complexity lower bound for a communication 
game in which the players hold two graphs on the same set of vertices. The task of the players is to find 
out whether the sets of vertices reachable from a specific vertex in exactly p + 1 steps intersect. The 
game requires a significant amount of communication only if the players are forced to speak in a specific 
difficult order This is reminiscent of lower bounds for communication problems such as indexing and 
pointer chasing. Among other things, our line of attack requires proving an information cost lower 
bound for a decision version of the classic pointer chasing problem and a direct sum type theorem for 
the disjunction of several instances of this problem. 
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1 Introduction 



Processing graph problems in the streaming model has attracted a fair amount of attention over the last 15 
years. Fomially, a streaming algorithm is presented with a sequence of graph edges and it can read them 
one by one in the order in which they appear in the sequence. The main computational resource studied for 
this kind of algorithm is the amount of space it can use, i.e., the amount of information about the graph the 
algorithm remembers during its execution. 

Furthermore, depending on the situation a small number of passes over the stream, instead of just one, 
may be allowed. This scenario may not be possible in the case of a router observing and analyzing high 
frequency network traffic, where the data is generated once and cannot be stored if it is extremely large. 
However, multiple passes over the stream may make sense in the case of processing a huge data set residing 
in a slow external storage device. For these kinds of devices, sequential access often maximizes readout 
efficiency and making a few passes over the data stream may be reasonable. 

At a more theoretical level, relations between nodes (i.e., how they are connected in the graph and what 
the distances between them are) are the essential property of graphs that is worth studying. When it comes 
to exploring the structure of graphs, allowing for multiple passes seems to greatly improve the capabilities of 
sti'eaming algorithms. For instance, the algorithm of Sarma, GoUapudi, and Panigrahy ll26l . which received 
the PODS best paper award, uses multiple passes to construct long random walks in the graph in order to 
approximate PageRank for vertices. Also many strong lower bounds of J7(n^) space for one pass easily 
break if more than one pass is allowed. This is for instance the case for the early lower bounds of Henzinger, 
Raghavan, Rajogopalan ifTSl and also the lower bounds of Feigenbaum et al. ifTTTl . 

On the other hand, constructing lower bounds for graph problems is usually based on constructing 
obstacles for local exploration, and our paper is not different in this respect. We show that finding out if two 
vertices are at a specific small distance p essentially requires p/2 passes to be accomplished in space 0(n). 
The main idea is similar to what is done for pointer chasing. Namely, we place edges in the order opposite 
to the sequence which enables easy exploration. 

Our results. Let n be the number of vertices in the graph and let p be the allowed number of passes. We 
show strongly superlinear lower bounds of T„ p = Q, ^^^^ space for three problems: 

• testing if the graph has a perfect matching, 

• testing if two prespecified vertices u and v are at distance at most 2(p + 1) for an undirected input 
graph, 

• testing if there is a directed path from u and v, where u and v sue prespecified vertices and the input 
graph is directed. 

In general, lower bounds stronger than Q{n) require embedding a difficult instance of a problem into the 
"space of edges" as opposed to the "space of vertices," which turns out to be difficult in many cases. For 
instance, the Q{n?) lower bounds of iflSl and iflTI do not hold for algorithms that are allowed more than 
one pass. We describe our hard communication problem from which we reduce to the streaming problems 
in Section [21 We now overview related work. 

Matchings. In the maximum matching problem, the goal is to produce a maximum-cardinality set of non- 
adjacent edges. Streaming algorithms for this problem and its weighted version have received a lot of 
attention in recent years |[n]|22]|9l[I]|2l]|l3l|l9l[l0]|271. 

Our result compares most directly to the lower bound of Feigenbaum et al. ifTTTl . who show that even 
checking if a given matching is of maximum size requires Q{n?) space in one pass. Our result can be seen 
as an extension of their lower bound to the case when multiple passes are allowed. Even when p>2 passes 
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are allowed, we show that still a superlinear amount of space, roughly required to find out 

if there is a perfect matching in the graph. This of course implies that tasks such as computing a maximum 
matching or even simply the size of the maximum matching also require this amount of space. 

For the approximate version of the maximum matching problem, McGregor |[22l showed that a (1 — e)- 
approximation can be computed in 0{n) space with the number of passes that is a function of only e. The 
only known superlinear lower bound for the approximate matching size works applies only to one-pass 
algorithms and shows that the required amount of space is n^+^(^/ ^"^s^ogn) ^ constant approximation 
factor better than 1 — e^^ is desired |[T3l[T9l . 

Shortest paths. We now move to the problem of computing distances between vertices in an undirected 
graph. Feigenbaum et al. lfT2l show that 0{n) space and one pass suffice to compute an 0(log n/ log log n)- 
spanner and therefore approximate all distances up to a factor of 0(logn/loglogn). They also show a 
closely matching lower bound of fl{n^~^^^^) for computing a factor t approximation to distances between all 
pairs of vertices. 

In the most closely related result to ours, they show that computing the set of vertices at distance p from 
a prespecified vertex in less than p/2 passes requires n^+^i^/P^ space. In this paper, we show a similar 
lower bound for the problem of just checking if two specific vertices are at distance exactly p. Our problem 
is algorithmically easier, as can be shown by first improving their lower bound to show that their lower 
bound holds even when the number of allowed passes is p — 1. (This follows by replacing one of their 
proof components with a stronger pointer chasing result from |[T4l .') As a result, to compute the distance p 
neighborhood, essentially the best thing one can do is to simulate the BFS exploration with one step per pass 
over the input. For our problem of checking if two vertices are at distance p, p/2 passes and 0{n) space 
suffice, because one can simulate the BFS algorithm up to the radius of p/2 from both vertices of interest. 
This is one of the reasons why our result cannot be shown directly by applying their lower bound. 

A space lower bound of il(n^) for one pass algorithms to find whether a pair of nodes is at distance 3 
can be found in lITTI . 

Directed connectivity. Feigenbaum et al. ifTTI show that the directed u-v connectivity problem requires 
Q{n'^) bits of space to solve in one pass. However, their lower bound does not extend to more than one 
pass. Once again our lower bound extends their result to show that a superlinear lower bound holds for 
multiple passes. (Note that the problem of connectivity can easily be solved with one pass and 0{n) space 
for undirected graphs, using for instance the well known union-find algorithm.) 

Paper organization. We begin in Section |2] with a description of the communication problems we study 
and a high-level overview of our lower bound approach. We set up some useful information-theoretic pre- 
liminaries in Section |3] We state our main communication complexity lower bound (Theorem |4]i and use it 
to show our streaming lower bounds in Section |4] Our communication lower bound is proved in three steps, 
and we go into the details of these steps in the next three sections. Finally, in Section [8] we put the steps 
together to give a proof of Theorem @] 

2 Proof overview and techniques 

Via simple reductions, our multiple-pass streaming lower bounds for matching and connectivity reduce to 
proving communication complexity lower bounds for a certain decision version of "set pointer chasing." 
The reductions to streaming ai^e described in Section 5] and in this section we give an overview of our 
communication complexity results. We start with a description of the communication problems that will be 
useful for us. 
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2.1 Communication Problems 



We assume private randomness in all communication problems, unless otherwise stated, and all messages 
are public. Consider a communication problem with p players Pi, . .. , Pp. Players speak in r rounds and in 
each round they speak in order Pi through Pp. At the end of the last round, the player Pp has to output the 
solution. We call any such problem a {p, r) -communication problem. 

We define [n] as {1, ... , n}. For any function f : ^(^)» we define a mapping / : V{A) — V{B) 
such that /(5) = Ue5/(«)- 

Pointer and Set Chasing. The pointer chasing communication problem PC„^p, where n and p are positive 
integers, is a (fc, A; — 1) -communication problem in which the i-th player Pi has a function fi : [n] — )• [n] 
and the goal is to compute /i(/2(. • • /fc(l) • • •))■ 

The set chasing communication problem SC„^p, for given positive integers n and p, is a {p,p — 1)- 
communication problem in which the i-th player Pi has a function fi : [n] — )• 'P([n]) and the goal is to 
compute /;({!})...)). 

Operators on Problems. For a {p, r)-communication problem X, we write EQUAL(A') to denote a (2p, r)- 
communication problem in which the first p players Pi, ... , Pp hold one instance of X, the next p players 
Pp+i, . . . , P2p hold another instance of X, and the goal is to output one bit that equals 1 if and only if the 
outputs for the instances of X are equal. See Figure [Jfor an example. 




Pi Pi Pi Pi A Pg P- Ps 



Figure 1: A sample instance of EQUAL(PC7^4) with a negative solution. It consists of two instances of 
PCj^4 held by two different sets of players. 

Analogously, for a {p, r) -communication problem X such that the output is a set, we write INTERSECT(Af ) 
to denote the {2p, r)-communication problem in which the first p players Pi, . .. , Pp hold one instance of 
X, the next p players Pp+i, . .. , P2p hold another instance of X, and the goal is to output one bit that equals 
1 if and only if the sets that are solutions to the instances of X intersect. See Figure |2]for an example. 

For a (p, r)-communication problem X with a Boolean output, we write ORf(Af), where t is a positive 
integer, to denote the (p, r)-communication problem in which players have t instances of X and want to 
output the disjunction of their results. 

Limited Pointer Chasing Equality. We say that a function / : A — i? is r-non-injective, where r is a 
positive integer, if there is A' C A of size r and b ^ B such that for all a G A' , f{a) = b. 

We write LPCE„ to denote a modified version of EQUAL(PC„_^:). In LPCE„ ^ ^ the last player has to 
output the same value as in EQUAL(PC„^fc), unless one of functions in one of the pointer chasing instances 
is t-non-injective, in which case the last player has to output 1. This is a technical extension to ensure that 
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Figure 2: A sample instance of INTERSECT(SC7^4), where two final sets intersect. The edges outgoing 
from vertices that are not visited were omitted. 

no element has too many pre-images and is needed for our reduction between INTERSECT(SCn,p) and 
OR,(EQUAL(PC„,p)). 

2.2 Lower bound for INTERSECT(SC„,j,) 

Our multiple-pass streaming lower bounds for matching and connectivity reduce to proving communication 
complexity lower bounds for the set chasing intersection problem INTERSECT(SC„^p). Note that if the 
players spoke in the order P2p, P2p^i, ■ ■ ■ , Pi, then they would be able to solve both instances of SCn,p, 
using at most 0{n) communication per player, which is enough to solve the intersection problem. If the 
players spoke in the desired order Pi, P2, .. ., P2p but were allowed a total of p rounds then they would be 
able to solve the instances of SC„^j, with 0{n) communication per message by simulating one step in the 
pointer chasing instance per round. Our main result is however that if the number of allowed rounds is p — 1, 
then approximately n^+^(^/p) bits of communication are needed to solve the problem, even for randomized 
protocols with constant error. 

Our result is reminiscent of the classic n{n) communication complexity lower bounds for problems 
such as indexing and pointer chasing PCn,p l[24l[T4l when the players speak in the "wrong" order. Guha and 
McGregor |[T4l adapt the proof of Nisan and Wigderson ll24l to show that solving PC„^p (in p — 1 passes) 
requires Q{n)/p'^^^^ total communication even if the protocol can be randomized and can eiT with small 
constant probability. Increasing the number of rounds to p or letting the players speak in the opposite order 
(even in just one round) would result in a problem easily solvable with at most O(logn) information per 
message. 

Even more directly related is the construction of Feigenbaum et al. (12L who show that solving SC„^p 
requires n^+^(^/p) communication in less then p passed Their proof follows by using a direct sum theorem 
of Jain, Radhakrishnan and Sen lITSl to show that solving t n^C^/p) instances of PCn,p requires roughly 
t times more communication than solving a single instance. Then they show that an efficient protocol for 
solving SC„^p would result in an efficient protocol for solving t instances of PCn,p in pai^allel. 

Compared to SC„^p, INTERSECT(SC„^p) is a decision problem. In particular, there seems to be no 
reduction allowing to reconstruct the sets reached in INTERSECT(SC„^p). The only thing that we learn 
after an execution of the protocol is whether these two sets intersect. Therefore, reducing our question to 
that of |[T2l seems unhkely. 

'in fact, they show this for roughly less then p/2 passes, but replacing the lower bound of |24J with the lower bound of 1141 and 
extending some other complexity results to the setting with multiple players yields the improved bound claimed here. 
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Our proof of the above communication complexity lower bound proceeds in three steps: 

Step A: Reduction to proving a communication lower bound for ORt(EQUAL(PC„^p)). 

Step B : A direct sum style step lower bounding the communication complexity of ORt(EQUAL(PC„_p)) 
as roughly t times the communication complexity of EQUAL(PC„ p). 

Step C: An information complexity lower bound for EQUAL(PC„^p). 

The technical body of the paper actually proves these steps in the opposite order (Steps A,B,C are discussed 
in Sections |7J[6l Irrespectively). But here we will expand on the steps in the above order. The actual proof 
works with a variant of EQUAL(PC„^p), namely LPCE„ p which we defined earlier, in order to deal with 
functions /j that may be highly non-injective, and which may break the reduction in Step A. For simplicity, 
we ignore this aspect in the overview, but it is worth keeping in mind that this complicates the execution of 
Step C on the information complexity lower bound. 

Step A: Reduction to ORt(EQUAL(PC„^p)). Our idea here is to use a communication protocol for 
INTERSECT(SC„^p) to give a protocol that can answer if at least one of t instances of EQUAL(PC„^p) 
has a Yes answer. (A reminder that in the EQUAL(PC„^p) problem, the input consists of two instances of 
PC„^p with functions {/j, gi : [n] — )■ and the goal is to output Yes iff we end up at the same index in 

both instances, i.e., if /i(/2(. • • /p(l) • • •)) = 9i{92{- ■ ■ S'p(l) • • •))•) Given t instances of EQUAL(PC„,p), 
for each instance independently, we randomly scramble the connections in every layer while preserving the 
answer to EQUAL(PC„^p). We then overlay all these instances on top of each other to construct an instance 
of INTERSECT(SC„^p) (note each node has exactly t neighbors in the previous layer). 

By construction, given an Yes instance of ORt(EQUAL(PC„^p)), by following the mappings from the 
instance of EQUAL(PC„^p) which has an Yes answer, we also obtain an element that belongs to the inter- 
section of two resulting sets in INTERSECT(SC,j^p). Since t = 

n®Wp)^ we have t'^P <^ n, and we argue 
that the random scramblings ensure that if none of t instances of EQUAL(PC„^p) have an Yes answer, then it 
is unlikely that the two resulting sets in the instance of INTERSECT(SC„^p) will intersect. This constraint 
on t is what limits our lower bound to ~ 

Step B: A direct sum style argument. In this step, our goal is to argue that the randomized communication 
complexity of ORf(EQUAL(PC„^p)) is asymptotically times larger than that of EQUAL(PC„^p). This 
is reminiscent of direct sum results of the flavor that computing answers to t instances of a problem require 
asymptotically t times the resources, but here we only have to compute the OR of t instances. Our approach 
is to use the infonnation complexity method that has emerged in the last decade as a potent tool to tackle 
such direct sum like questions ||8l|2l[T8], and more recently in 131 El and follow-up works. The introductions 
of E [161 contain more detailed information and references on direct sum and direct product theorems in 
communication complexity. 

Our hard distribution will be the uniform distribution over all inputs. Being a product distribution, the 
information complexity will be at least the sum of the mutual information between the i-th input and the 
transcript, for 1 < i < t. Using the fact that the probability of an Yes answer on a random instance of 
EQUAL(PC„^p) is very small (at most 0(l/n)), we prove that the mutual information between the f'th 
input and the transcript can't be much smaller than the information cost of EQUAL(PC„^p) for protocols 
that err with probability o(l/n) under the uniform distribution. 

Step C: Lower bound for information cost of EQUAL(PC„^p). This leaves us with the task of lower 
bounding the infonnation cost of low error protocols for EQUAL(PC„^p) under the uniform distribution. 
This is the most technical of the three steps. We divide this step into two parts. 

First we show that if there is a protocol with low information cost IC on the uniform distribution, then 
there is a deterministic protocol that on the uniform distribution sends mostly short messages, and errs with 
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at most twice the probability. This is done by adapting the proof of the message compression result of lITSl 
for bounded round communication protocols. We cannot use their result as such since in order to limit the 
increase in error probability to 7, the protocol needs communicate il(l/7^) bits. This is prohibitive for us 
as we need to keep the enw probability as small as 0(l/n), and can thus only afford an additive 0(l/n) 
increase. We present a twist to the simulation obtaining a deterministic protocol with at most twice the 
original error probability. The protocol might send a long message with some small probability e and in 
other cases communicates at most 0(IC /e^) bits. In our application, we set e to be a polynomial in \/p. 

The second part is a lower bound for EQUAL(PC„.p) against such "typically concise" deterministic 
protocols. To prove this, we show that if the messages in the deterministic protocol are too short, then with 
probability at least 1/2, the protocol will have little knowledge about whether the solutions to two instances 
of pointer chasing are identical and therefore, will still err with probability Q.{l/n), which is significant 
from our point of view. The proof extends the lower bound for pointer chasing due to Nisan and Wigderson 
|[24l and its adaptation due to Guha and McGregor |[T4l . We have to overcome some technical hurdles as 
we need a lower bound for the equality checking version and not for the harder problem of computing the 
pointer's value. Further, we need to show that a constant fraction of the protocol leaves are highly uncertain 
about their estimate of the pointers' values, so that they would err with probability 0(l/n) (with 1/n being 
the collision probability for completely random and independent values). 

Summarizing, Step C can be seen as a modification of techniques of [24l [14] to prove a communication 
lower bound for EQUAL(PC„ p) combined with techniques boiTowed from flSl to imply a lower bound for 
information complexity. The relationship between information complexity and communication complexity 
has been a topic of several papers, starting with ||8] [18] for protocols with few rounds, and more recently 
|[l[5l[l[7l[6l[2l for general protocols. 

3 Preliminaries 

Constant d,. Let d, be a constant such that the probability that a function / : [n] — > [n] selected uniformly 
at random is C*(l + log n)-non-injective is bounded by l/{2n?). The existence of follows from a 
combination of the Chemoff and union bounds. 

3.1 Information and Communication Complexity 

An important part in our proofs is played by communication and information complexity, which we now 
define for completeness. 

The communication complexity of a protocol is the function from the input size to the maximum length 
of messages generated by the protocol on an input of a specific size. For a problem X and 5 G [0, 1], the 
communication complexity of X with error 5 is the function from the input size to the infimum communica- 
tion complexity of private-randomness protocols that err with probability at most 5 on any input. We write 
Rs{X) to denote this quantity. 

The information coi'illlCost^(n) of a protocol IT on input distribution ijj equals the mutual information 
I{X; Ii{X)), where X is a random variable distributed according to and Ii{X) is the transcript of 11 on 
input X. 

The information complexity IC^ ,5 of a problem A:" on a distribution -0 with error 5 is the infimum 

"Note that this is the external infonnation cost following the teiminology of [3]. For product distributions ip, this also equals 
the internal information cost. As product distributions will be our exclusive focus in this paper, this distinction is not relevant to us, 
and we will simply use the term information cost. 
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ICost^(n) taken over all private-randomness protocols 11 that eix with probability at most 6 for any in- 
put. 

3.2 Some useful information-theoretic lemmas 

Let us first recall a result that says that if a random variable has large entropy, then it is behaves almost like 
the uniform random variable on large sets. 

Fact 1 ( ll25l . see also |[24l Lemma 2.10]). Let X be a random variable on [n] with H{X) > log n — S. Let 

S C [n] and let A = ^J^. If A < 1/10, then Pv[X G S"] > ^ (1 - A) . 

Using the above result, we show that it is hai^d to predict with probability 1 — o(l/n) if two independent 
random variables distributed on [n] collide if they have large entropy. 

Lemma 2. Let X and Y be two independent random variables distributed on [n] such that both H{X) and 
H{Y) are at least logn — 6, where 6 = 48~^. Then 

• Pt[X = ¥]> l/(8n), and 

• ifn > \, Pr[X / y] > 1/4. 

Proof. We first prove that there is a set Sx Q [n] such that \ Sx\ and for each x S Sx, Pr[X = x] > 

l/(2n). Suppose that there is no such set. Then there is a set Tx of size more than ?i/4 in which every 

element has probability less than l/(2n), and therefore, Pr[X G Tx] < Note that sj-^ < 1/12 < 

1/10, which implies that we can apply Fact[I]to Tx- We obtain Pr[X € Tx] > • which contradicts 
the assumption that there is no such set. 

Analogously, one can prove that there is a set Sy ^ [n] such that \Sy\ > jn and for each y G Sy, 
Pr[y = y]> l/(2n). Note that \Sx n Sy\ > n/2. For each x e Sx n Sy, Ft[X = Y = x] > l/(4n2). 
Hence 

Pt[X = Y]> Yj Pr[X = y = x] > l/(8n). 

xeSxnSy 

To prove the second claim, for n > 4, observe that for every setting x of X, \Sy \ {x}\ > ^ — 1 > n/2, 
and therefore, the probability that y / X is at least 15*^ \ {x}| • l/(2n) > 1/4. □ 

The following gives a bound on the entropy of a variable that randomly selects out of two random values 
based on another 0-1 valued random variables. 

Lemma 3. Let Xq, Xi, and Y be independent discrete random variables, where Xq and Xi are distributed 
on the same set Q and Y is distributed o?i {0, 1}. Then 

H{Xy)<l + Y.U'^Ay = ^]-H{X,). 

Proof. For each i G {0,1} and each oj ^ Q., let = Pr[Xj = w]. Let q = Pr[y = 0]. For each 
— Lo] — qpo,uj + (1 ~ q)pi,uj- The inequality follows from the concavity of the function 
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f{x) = x- log(l/x) on [0, 1] and simple algebraic transformations: 



H{Xy) = ^ {qpQ^uj + (1 - <l)Pi,ic) ■ log (1/ {qpQ,^ + (1 - q)Pi,Lo)) 

< X] (^^0>'^ ' 1/9^*0,0;) + (1 - 9)Pl,a; • log(l/(l - q)pi,ui)) 
= 9 • log(l/g) + g • ^PO,a;log(l/po,a;) 

+ {l-q)- log(l/(l - g)) + (1 - • Pl,u. log(lM,a;) 

wen 

= ^(g) + (7-i^(Xo) + (l-g)-//(Xi) 

1 

4 The Main Tool and Its Applications 

The main tool in our paper is the following lower bound for the communication complexity of set chasing 
intersection. 

Theorem 4. We have the lower bound 

,l+l/(2p) \ 



i?i/io(INTERSECT(SC„,p)) = ^ 



n 



^pi6 . log3/2 n j ' 
provided n > "dip) for some sufficiently large function : N — > N. 

We now present relatively straightforward applications of this theorem to three graph problems in the 
streaming model. 

Theorem 5. Let p be a positive integer Solving the following problems with probability at least 9/10 in the 
streaming model with p passes requires at least Vt { ^20^1 3/2 ) ^'^■^ of space: 

Problem 1: For two given vertices u and v in an undirected graph, check if the distance between them 
is at most 2{p + 1). 

Problem 2: For two given vertices u and v in a directed graph, check if there is a directed path from u 
to V. 

Problem 3: Test if the input graph has a perfect matching. 

Proof Let us consider the problems one by one. For Problem 1, we turn an instance of INTERSECT(SCfc p+i] 
into a graph on n = {2p + 3) • /c vertices. We modify the graph in Figure |2] First, we make all edges undi- 
rected. Second, we merge every pair of middles vertices connected with a horizontal line into a single 
vertex. Any path between the top leftmost vertex u and the top rightmost vertex v is of length at least 2p + 2. 
The length of the path is exactly 2p + 2 if and only if it moves to the next layer in each step. Note that 
this corresponds to the case that the final sets for two instances of SC^ p+i intersect. We create the input 
stream by inserting first the edges describing the function held by Pi, then by P2 and so on, until P2p+2- 
If there is a streaming algorithm for the problem that uses at most T bits of space, then clearly there is 
a communication protocol for INTERSECT(SCfc,p+i) with total communication (2p + 2) ■ p ■ T and the 
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Figure 3: Reduction to the perfect matching problem. 



same eiTor probability as the streaming algorithm. The protocol can be obtained by the players by simu- 
lating the streaming algorithms on their parts of the input and communicating its state. This implies that 

rp_c(i (n/py+ymp+i)) ^ _ ^ ( ^i+i/(2(p+i)) \ 

Vp^" pi6.1og3/2n ) ~ \ p20-log='/2„ )■ 

For Problem 2, the reduction is almost the same, with the only difference being that we make all edge di- 
rected from left to right and we want to figure out if there is a directed path from the top leftmost vertex to the 
top rightmost vertex. Such apath exists if and only if the final sets in the instance of INTERSECT(SCjt,p+i) 
intersect. 

For Problem 3, the reduction is slightly more complicated. We show how to modify the hard instance G 
that we have created for Problem 1. Let us first add a perfect matching before and after every layer of edges 
of the instance hard for Problem 1 , except for the first and the last layer, in which we omit one edge. The 
omitted edges are incident to the vertices u and v corresponding to value 1, i.e., the vertices that we want 
to connect using a path going directly from left to right in Problem 1 . See Figure [3] for an example. Note 
that the additional edges constitute a matching M in which all but two vertices are matched. Now the graph 
has a perfect matching iff there exists an augmenting path in M between u and v, which are the unmatched 
vertices. Any augmenting path has to alternate between matched and unmatched edges, which implies in our 
case, that it has to go directly from left to right. Therefore, any augmenting path in M corresponds to a path 
going directly from left to right in G and connecting u and v. The only difference is that the augmenting path 
has additional edges coming from the matchings were inserted into G. Therefore the streaming algorithm 
for testing if a graph has a perfect matching can be used to create a protocol for INTERSECT(SCfc p+i), 
where the additional edges can be placed at the beginning of the stream. □ 



5 Step 1: Information Complexity of Pointer Chasing Equality 

To prove the main theorem of the paper, we first show a lower bound for the information complexity of 
Limited Pointer Chasing Equality. 

Lemma 6. Let n and p be positive integers such that n > 18p^. It holds 

Tr n PPF N ^ " [2 log n] ^_^fn\ ^f^ogn \ 

i<-M,l/{64n)ll-rUt„^p^C'.(l+logn)j ^ ^wTs^TplG ~ 2l3 . 32 . p8 " ^ " ^ye ^ " ^ J ' 

where /x is the uniform distribution on all possible inputs of the problem. 

The proof consists of two smaller steps. First we show that if there is a protocol with low information 
cost on the uniform distribution, then there is a deterministic protocol that on the uniform distribution sends 
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mostly short messages, and errs with at most twice the probability. Then we show that the messages in the 
deterministic protocol cannot be too short, because with probability at least 1/2, the protocol will have little 
knowledge about whether the solutions to two instances of pointer chasing are identical and therefore, will 
still eiT with probability significant from our point of view. 

5.1 Transformation to Deterministic Typically Concise Protocols 

Let us first define concise protocols, which send short messages most of the time. 

Definition 7. We say that a protocol P is an (m, e)-concise /or an input distribution fi if for each i, the 
probability that the i-th message in the protocol is longer then m is bounded by e. 

The following three facts are from fTf] [TSl and regard regarding infomiation theory and random vaii- 
ables. They are useful in our proofs. 

Fact 8 (HU Fact 1]). Let X, Y, and Z be random variables. The following identity holds: I{X; Y, Z) = 
I{X;Y)+IiX;Z\Y). 

Fact 9 ( II18I Fact 2]). Let X and Y be a pair of random variables. Let P be the distribution ofY and let P^ 
be the distribution ofY given X = x. Then I{X;Y) = Ex[DKh{Px\\P)\ where -Dkl(Qi||<32) denotes 
the KL-divergence between distributions Qi,Q2- 

Fact 10 ( ifT/l Substate Theorem]). Let P and Q be probability distributions on N such that Dkl{P\\Q) = o- 
Let e G (0, 1) and let Good = {i E N : Pr(i) • 2-("+i)/= < Q{i)]. Then Vt[X G Good] > 1 - e. 

We now show an auxiliary lemma that shows that if Dyj^{P\\Q) is bounded then a relatively short 
sequence of independent random vaiiables distributed according to Q suffices to generate a random value 
from P. The lemma is an adaptation of a lemma from lITSl . 

Lemma 11. Let P and Q be two probability distributions on such that D]<j^{P\\Q) < oo. Let {Ti,T2,Ts, . . .) 
be a sequence of independent random variables, each distributed according to Q. Let Fq = — 1. Let 
e G (0, 1). There is a set Good C N and a random variable i? € N such that 

• I^ieGood P^) > 1 - e. 

• for all X G Good, Pr[r/j^ = x] = P{x), 

• E[R] < 2(^kl(p||Q)+i)/£. 

Proof Let the set Good be defined as in Fact[lOl i.e.. Good = {i G N : P{i) ■ 2-("+^)/^ < Q{i)}, where we 
set a = Dkl{P\\Q)- Following ifTSl . we use rejection sampling to prove the lemma. Consider the following 
process. For consecutive positive integers j, starting from 1, do the following. Look at the value 7^ taken 
by Fj. If 7j G Good, toss a biased coin and with probability P{j) ■ 2~("+^)/^/Q(j), set R = j and finish 
the process. If 7j Good or the coin toss did not terminate the process, toss another biased coin and with 
probability 

2-(a+l)/e (1 - E»gG00d-P(^)) 

(l-2-("+i)/-E.eGoodn^))' 
set = and also terminate the process. Otherwise, continue with j set to j + 1. The process terminates 
with probability 1. 

Let us argue that R and Good have the desired thi^ee properties. The first property is a consequence of 
Fact [TOl To prove the other two, observe what happens when the process reaches a specific j and before 
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it moves to the next j or terminates. The process terminates with R = j and Tn = x for a specific 
X G Good with probability P{x) ■ 2"'^'^+^^/'^. The probability that it terminates with R = equals exactly 
ieGood ^(^))- Since these probabihties are independent of j, when the process eventually 
terminates the probability of Fr = x for each x G Good is exactly P{x), which proves the second property. 
Finally, the probability that the process terminates for a specific j after reaching it is exactly 2~("+^)/^. 
Clearly, E[R\ is bounded from above by the expected j for which the process stops, which in turn equals 
exactly 2^''+^^^. □ 

The following lemma allows for converting protocols with bounded information cost on a specific dis- 
tribution into deterministic protocols that mostly send short messages on the same distribution. The proof 
of the lemma is a modification of the message compression result of lITSl . An important feature of our ver- 
sion is that the error probability is only doubled, instead of an additive constant increase which we cannot 
afford. A simple but key concept we use to achieve this is to allow the protocol to send long messages with 
some small (constant) probability. We then handle such "typically concise" protocols in our lower bound of 
Section [ 



Lemma 12. Let H be a private-randomness protocol for a (p, r) -communication problem V such that H 
errs with probability at most 6 > on a distribution ^. For any q > 0, there is a protocol H' for V such that 

• n' is deterministic, 

• n' errs with probability at most 25 on fi, 

• n' is (m, q)-concise, where m = 128 • (ICost^(n) + 2) • (pr/q)^. 

Proof. We start with a proof ovei^view. There are. pr — 1 communication rounds in IT. We construct a series 
of intennediate protocols Yl'^^^i, npr-2' ■ ■ ■ > n'^^, where IT'- is a modification of the protocol 11 that is likely 
to send a short message in rounds i, i -\- 1, . . . , pr — I. In the first i — 1 rounds, 11^ behaves in the same way 
as n. In particular, I[[ uses only private randomness to generate the first i — 1 messages. In later rounds, 
the protocol uses public randomness. The players in the modified protocols will convey exactly the same 
information as the original protocol 11 and therefore, the protocols will err with the same probability, with 
the only difference being a different encoding of messages and the use of pubUc randomness. 

For convenience, let 11^^ be the original protocol IT. We now explain how we convert n^_|_i into 11^. Let 
Mq be the random variable corresponding to sequence of the first i — 1 messages in n'j^j. Let Mi be the 
random variable describing the i-th message in n^^^^. Let M = (Mq, Mi). Recall that M is distributed in 
the same way as its equivalent for the original protocol P. Let Pj be the player sending the i-th message 
(i.e., j = i (mod p)). Let X be the combined inputs of the other players, and let Y be the input of the j-th 
player. We write A/"'" to describe the distribution of Mi when Mq = mo. Moreover, we write M^'^'™" to 
describe the distribution of Mi when X = x, Y = y, and Mq = itiq. The distribution M^'^'^° does not 
depend on x, because the protocol uses only private randomness and to generate the i-th message it only 
uses the previous messages and y, the input of the i-th player. It follows from Fact|8]and Fact|9]that 



I{X, Y; M) = I{X, Y; Mq)+Em, [I{X, Y- Mi |Mo)] = I{X, Y- M^)+Em,,x,y S ( Mj^'^'*''" 



We define as -Ej\/o,.Y,y S ^Mj^'^'^° Mf^"j for this specific setting of i. Overall, it follows by in- 
duction that the mutual information between the input and the protocol transcript, i.e., ICost^j(P), equals 
Efli^ flj. This also imphes that S (Mf ||M™°) < oo for any setting X = x,Y = y, and Mq = tjiq 
that has nonzero probability. 

Recall that the first i — l messages of II'- are generated in the same way as in Il'j^i. We now describe how 
Pj generates the i-th message. Let mo be the messages sent so far. The distribution of the i-th message, M™" 
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is known to all the players. Let (Fi, Ts, . . .) be an infinite sequence of independent random variables, 
where each Fj is drawn independently from the distribution Af™°. The sequence of Fj's is generated using 
public randomness, so it is known to all the players as well. We now use Lemma[TT] where we set Q = M™° , 
P = and e = q/'&pr. Note that the distribution M^'^'™° does not depend on x, because the 

randomness is private in the first i messages in H'^^^. Pj will transmit exactly the same information in II'- as 
in n'j^^. The player Pj fixes a set Good and a random variable R as in Lemma [TT] If G Good, then the 
player sends a single bit followed by a prefix-free encoding of the value R. Due to the concavity of the 
logarithm, the expected length of the message can be bounded by 1 + lQpr{DY^i^{M^'^'^" ||M™") + l)/q + 
2 < 16pr(i:'KL(Mf'^'""'||Mf'°) + 2)/q, where the additional factor of 2 and the additional additive term 
of 1 come from the prefix free encoding. Overall, the expected length of the message starting with equals 
16pr{ai + 2)/q. 

If Tji Good, the player generates the message from the part of distribution M^'^'"^° restricted to 
N \ Good and transmits the selected value prefixing it with a single bit 1. Overall, all players can decode a 
message generated according to M^'^'™° and then behave in the same way as in the protocol ^i^i- 

After applying a sequence of pr — I steps of the transformation, we obtain a randomized protocol II'j^ 
that still errs with probability 6. We now show that there is a suitable setting of random bits in the protocol 
to obtain the desired deterministic protocol 11'. First, let Z G {0, 1} be a random variable indicating that the 
algorithm returned an incorrect answer. We have E[Z] = 6. It follows from Markov's inequality that the 
probability that fixing the random bits makes the algorithm eiT with probability higher than 26 on ^ is at most 
1/2. Consider now the i-the message in 11'^, where 1 < i < pr — 1. Let Z- be the random indicator variable 
that equals 1 if the i-th message starts with 1. It follows from our construction that E[Zl] < e = q/8pr. If we 
fix the random bits of the protocol, the probability that the i-th message starts with 1 with probability higher 
than g/2 is bounded by Markov's inequality by I /Apr. Finally, let Z'-' be the random indicator variable that 
the i-th message in starts with and has length greater than 128(aj + 2){pr/q)'^. Consider now a random 
variable Wi that equals the length of the i-th message if the message starts with and if it starts with 1 . 
We know that £"[^4] < Wpr{ai + 2)/q, and therefore, by Markov's inequality E[Z"] < q/8pr. Applying 
Markov's inequality again, for a random settings of the protocol's randomness, the expectation becomes 
larger than q/2 with probability at most l/Apr. Summarizing, by fixing the protocol's random bits, with 
probability at least l-l/2-l/(4pr)-(pr-l)-l/(4pr)-(pr-l) = l/2-(pr-l)/(2pr) > 1/2-1/2 = 0, 
we obtain a deterministic protocol that errs with probability at most 26, and sends a message longer than 
128(aj + 2){pr/q)'^ in the i-th round with probability bounded by q. The final claim follows from the fact 
that all a-i are bounded by ICost^(n). □ 

5.2 Lower Bound for Deterministic Typically Concise Protocols 

In this section, we show that a deterministic concise protocol for the Limited Pointer Chasing Equality 
cannot send short messages very often, unless it errs with probability Q.{l/n). The proof follows along the 
lines of the lower bound for Pointer Chasing due to Nisan and Wigderson ll24l and its adaptation due to Guha 
and McGregor |[T4l . The main technical differences come from the fact that we want to show a lower bound 
for Limited Pointer Chasing Equality. First, this requires ruling out the impact of the easy case when one of 
the functions is t-non-injectve for large t. Second, this requires showing that with constant probability, the 
last player is unlikely to know what the solutions to the input instances are, and since they are independent, 
they will collide with probability Q{l/n). 

Lemma 13. If > 18p^, then any deterministic {m,q)-concise protocol for LPCE„ p ,^^(i_,^iog„), where 
m < en/(4p^) — [21ogn], q = and e = {48p^)~^, errs with probability at least l/(16n) on the 

uniform distribution over all possible inputs. 
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Proof. Recall that in the ]-PCE.j^p c^[i+\ogn) problem, there are 2p players Pi, . . . , P2p, with players Pi 
and Pp+i, 1 < i < p, holding functions /j : [n] — )■ [n] and gi : [n] — )• [n], respectively. The goal of the 
problem is to output "1" if one of the functions is (7^,(1 + log n)-non-injective or /i(/2(- • • /^(l) • • •)) = 
51 (52 (• • • 5p(l) • • •))• OtheiAvise, "0" is the correct output. The players speak in order Pi ttu'ough P2p, p — 1 
times overall. 

Let Op+i = 1 and by induction, let = /j(aj+i) for each z G [p]. Analogously, let 6p+i = 1 and let 
= 9i{h+i) for each i G [p\. Unless one of the functions is C^(l + log n)-non-injective, the goal of the 
problem is to find out whether oi = 61. 

We make two modifications to the protocol: 

1. We augment the (m, g)-concise protocol by simulating in parallel the following natural protocol. 
Initially, we append the pair (0^+1,6^+1) to each message until we reach the player Pp, who can 
compute ttp = /p(ap+i). Then we pass the pair (ap,6p+i) until it reaches P2p, who can compute 
bp = gp{bp+i) and pass (op, bp) to the next player. In general, whenever a message (oj, bi) reaches 
Pi-i, it is updated to (aj_i,6i), and whenever a message (aj_i,6i) reaches Pp+j.i, it is updated 
to (aj_i,6i_i). This protocol finally computes (02,62). Appending the information, increases the 
length of each messages by [2 log n] . This way, we obtain a deterministic (m + [2 log 7i \ , -concise 
protocol. 

2. Additionally, the first time a player is reached in the protocol, if the player's function is C*(l + log n)- 
non-injective, we make the player send a message longer than m + [2 log n] bits. This may require 
modifying the behavior of the protocol in some other situations. 

Recall that the player's function is Cj,(l + logn)-non-injective with probability at most l/(2n^). If 
the player is already sending long messages in some cases, we can relabel and extend some of the 
long messages to accommodate the transition of some short messages into long messages. In this case 
the probability of sending a long message increases by at most l/(2n^). 

Likewise, if one of at least prefixes of length [2 log n] is not used by the protocol at all, we can 
use this prefix to transmit long messages. In this case, the probability of a long message increases by 
at most l/(2n^) as well. 

Finally, if the player does not send long messages in this case and all prefixes of length [2 log n] are 
used, one of these prefixes occurs with probability at most 1/n^. We extend all messages starting with 
this prefix to turn them into long messages and reduce the case to first one. The extension increases 
the probability of long messages by at most 1/n^. 

Overall, the probability of long messages can increase by at most l/(2n^) + < 2/n?. As a result 
we obtain a deterministic (m + [2 log n\,q + 2/?7,^) -concise protocol. 

Let m' = m + \2 log n] and q' = q + 2/n?. 

From now on we think of our deterministic protocol as a decision tree of depth 2p{p — 1). The z-th layer 
of nodes, 1 < i < 2p{p — 1), corresponds to the situation when the control is passed to the player Pj, where 
j = i (mod 2p). Each leaf in the tree is labeled with either a "0" or a "1", corresponding to the decision 
made by the algorithm. Each edge outgoing from nodes at layers 1 through 2p{p — 1) — 1 is labeled with 
the message that the corresponding player sends, given her input and the previous messages. Edges between 
the last two layers are not labeled, because the last player does not send a message. 

We now introduce a few definitions for each node z in the decision tree: 

• Cz'- We set Cz to the total length of messages sent on the path from the root to z. 
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• Ff X ■ ■ ■ X Fp X Gf X . . . X Gp'. Let be the set of all functions from n to n. Since the protocol is 
deterministic, for each node z, the set of input functions {fi, fp, gi, gp) for which the protocol 
reaches z can be described as a product Ff x ■ ■ ■ x F^ x Gf x . . . x Gp CI J^^p Note that if the node 
is reached then the probability of each tuple in x • • • x x Gf x . . . x is identical. 

• iz and j^: We make iz and jz be the indices of the last pair (oj^ , bi, ) sent on the path from the root to 
z. For the root we assume that the pair is (a^+i, = (1, 1), i.e., iroot = jroot = p + I- Recall 
that for all z,iz > 2 and jz > 2. 

• {Az,Bz:): {Az,Bz) is a pair of random variables. Its random value is generated by selecting indepen- 
dently uniformly at random two functions e ^i^^i 9* ^ ^^'^ ^PPlyirig them to (oj^ , bjj 
to obtain {fi,{ai,), gi,{bjj). {Az,Bz) describes the possible values of the pair {ai,bj) if we move 
one step ahead in applying functions fi and gj, compared to the trivial algorithm that we simulate in 
parallel. Since the protocol is deterministic, the inputs of players are independent, and and Bz 
depend on inputs of disjoint sets of players, the variables Az and Bz are independent. 

We say that a node z is confusing if it has the following properties: 

1. All messages sent on the path to z have length bounded by m'. 

2. zisa leaf or for all i G [n], both \Ff\ > 2"^'^^ | J"| and |Gf | > 2"^''^ \T\, where is the total length 
of messages on the path from the root to z. 

3. H{Az) > log n - 5 and H{Bz) > log n - 5, where 5 = e^/^. 

It is easy to see that the root of the decision tree is confusing. We now prove by induction that the 
probability that for a random input, the protocols reaches a non-confusing node in step i is bounded by 
{i — 1) ■ {q + + 4e^/^). Suppose that the claim is true for step i and let us prove it for step i + 1. We 
bound the probability that a specific property is violated. 

1. The probability that the first property is violated is bounded by q', because the protocol is {m', q')- 
concise. 

2. Consider a confusing node z in step i. If i = 2p{p — 1), the children of z are leaves, and the property 
holds. So it suffices to focus on the case that i < 2p{p — 1). What is the probability that the second 
property is violated for some w of zl Let Pj be the player in control of step i. Without loss of 
generality, let us assume that 1 < j < p. Note that for all t S [p], Gf = Gf and for all t G [p] \ {j}, 

= F^. The property may only be violated for F^'. For each child w of z, let = — c^. Let 
W be the random variable representing the distribution of children of z. It holds 



Pr 



< Pr 



\F, 



1^ 



where the second to last inequality follows from the fact that > 2 log n, and the last inequality 
follows from Kraft's inequality. Therefore the probability that a confusing node loses the second 
property in the next step is bounded by 

3. It remains to bound the probability that the third property is lost. Let z be a confusing node in 
step i and let Pj be the player in charge of this step. If neither j = iz — 1 nor j — p = jz — 1> 
then for any child w of z, F^"'_i = and GJ _i = In this case the pairs of variables 

{Az,Bz) and {Au,, B^) have the same distribution and therefore the respective entropies remain the 
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same. Consider now the case that j = iz — 1- Pj computes aj = fj{ai^) and we need to bound 
the entropy H{Aw) of Au, for all children w of z, which is essentially the entropy of aj-i given all 
the information communicated so far. The information about fj^i at each child w can be expressed 
as a vector fJL^ = {f^_^{l), f^_-^{2), . . . , ff_^{n)) of random variables in [n]. We have H{Au,) = 
H{ff_^{aj)). Moreover, it holds 

Y.H{ff_,{t)) > H{ff_,)=\og\Ff_,\=\og\F^_,\ 

> log(2-2^--|J'|) = log I J"! - 2cz > n(logn - e). 

The first inequality above follows from subadditivity of entropy. The second and third inequalities 
follow from the fact that the function is uniformly distributed on Fj"_^ = Fj_^ of size bounded by 
the fact that z is confusing (Property 2). Finally the last inequality follows from the fact that z is 
confusing (Property 1), and therefore, en/2 > 2p{p — 1) • m' > Cz- 

For t uniformly distributed on [n], by Markov's inequality, we have 

Pr[F(/7„i(t))<logn-5]<e/5. 

Unfortunately, aj may not be uniformly distributed. However, we can exploit its high entropy, at least 
log n — 6. We apply Fact[T] Let S be the set of t such that H{f"^_^{t)) > logn — 5. We akeady know 
that 1 5*1 > (1 — £/5)n. Note that we can apply Fact[T] because 



A = J-^<J-^<V8~6<V8- e'/' < V8 • 48"^ < 1/10. 

\ \s\/n yi-f 

The probability that aj belongs to 5 is at least 

> 1 - (1 + \/8)e^/^ > 1 -4e^/^. 
Pr[//(/"Li(a,)) < logn - 6] < Ae^/^. 

n : ^ 




This implies that 



The case that j — p = — 1 is analogous, and therefore, the probability that the third property is lost 
in the next step is bounded by 4e^/^. 

Summarizing, the probability that moving from step i to step i + 1, we move from a confusing node to a 
non-confusing node is bounded by q' + 1/v? + Ae^/^ = Q + + Ae^/^ , which finishes the inductive 
proof. 

Overall, it follows that the protocol finishes in a non-confusing leaf with probability bounded by 2p{p — 
1) • (g + + 4eV3) <2p'^-q+'^ + Sp^- (48p2)-i < 1/6 + 1/6 + 1/6 = 1/2. 

Consider now a confusing leaf z. Recall that we modified the protocol so that if one of the functions 
is C^(l + logn)-non-injective a message longer than m' is transmitted. By definition, in such a case, the 
simulation of the protocol leads to a leaf that is not confusing. Therefore, the correct solution to an input 
instance that leads to z is solely based on whether ai = bi. We know that the random variables Az and 
Bz, which model ai and hi, respectively, are independent and both have entropy at least logn — 5, where 
5 < 48"^. Observe also that n > \/l8p^ > A. Hence it follows from Lemma|2]that whatever solution the 
protocol claims at z, be it "0" or "1", the claim is incorrect with probability at least l/(8n). Overall, on all 
inputs the protocol has to err with probability at least ^ • = □ 
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5.3 Proof of Lemma |6] 



We now combine Lemmas [12] and [T3] to prove Lemma |6l the main result of Section |5] 

Proof of Lemma^ Consider any protocol protocol IT for LPCE„ ^ (^^(^.j.iog^j that errs with probability 
at most l/(64n) on /x. By Lemma [T2l there is a deterministic (m, l/(12p ))-concise protocol IT' for 
LPCE„ p (^^(•i_i_iog„) that errs with probability at most l/(32n) on ^, where m = 128 • (ICost^(n) + 2) • 

( (i/gp^)) ) ' = 213 . 32 .p8 . (ICost^(n) + 2). It follows from Lemma[l3]that m > ^gp^ - [2 log n] = 

2i6.33.p8 - [2 log n] . Therefore, ICost^(n) > ^ij'j.a".^!^ - 1^^^ "2. Since this bound holds for any proto- 
col n that is correct with probability 1 — l/(64n), this is also a lower bound for the information complexity 
of the problem. □ 



6 Step 2: Direct Sum Theorem for Pointer Chasing Equality 

The following lemma is the main result of this section. 

Lemma 14. Let n, p, and t be integers such that v? > ISp^ and t < n/A. Let r = C*(l + log n). It holds 

«i/3(0R*(LPCE„,p,)) = 17 (^Te^) - O [pt^) . 

Before we prove it, let us first recall two classic results. First, the information complexity is a lower 
bound for randomized communication complexity of a protocol that errs with the same probability. 

Lemma 15 (lH Proposition 4.3]). Let 5 G (0, 1). For any communication problem V and any distribution 
Tp on inputs, RsiV) > IC^,5('P). 

Second, if the input distribution is a product distribution on multiple instances of a subproblem, then the 
total information revealed by the protocol transcript equals at least the information revealed for each of the 
instances. 

Lemma 16 (HJ Lemma 5.1]). Let V be a communication problem with Boolean output and let U be a 
private-randomness protocol for ORt{V)for a positive integer t. Let ipbe a distribution on inputs ofV. Let 
X = (Xi, . . . , Xt) be a vector of independent random variables with each distributed according to ip. For 
any input x, let Yi{x) be the transcript ofli on x. Then the following inequality holds: 

/(X;n(x))>E-=iA^-n(x)). 

Now we show the main ingredient, which is a proof that any correct protocol for OR((LPCE„ p^^ ) has 
to reveal almost as much information about each coordinate as if it was separately solving the corresponding 
instances of LPCE„ p ,.. 

Lemma 17. Let II be a private -randomness protocol for OR^ (LPCE„ p r) that errs with probability at most 
6. Let X = {Xi, . . . ,Xt) be a random vector with each coordinate Xi independently selected from the 
uniform distribution fi on all possible inputs to LPCE„ p Let Il{x) be the transcript o/H on input x. 
Ifr>C-i,- (log n + 1), t < n/4, and p < n, then for each i £ [t], it holds 

I{Xf,U{X)) > IC^,25(LPCE„,p,,) - Aptlogn- 1. 
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Proof. For each j G [t], let Yj G {0, 1} be the the solution to LPCE.„ p ^ on a specific coordinate Xj. The 
probability that Y,- = 1 is bounded by 1/n + 2p • l/{2v?) = 2/n, where the first term comes from the 
probability that the equality of two instances of PC„^p holds and the other is a bound on the probability that 
one of the functions is r-non-injective. 

Fix i G [t\. By the union bound, the probability that \] j^{Yj = 1 is bounded byt-2/n < 1/2. 
If \l j^^Yj = 0, the solution to ORf(LPCE„,p,r) on the input instance equals Yi. Therefore, 11 has to 
compute Yi with probability at least 1 — 25, provided Vjyi ^ = 0- If it erred with higher probability, it 
would overall err on the input instance of ORt(LPCE„ p r) with probability greater than 6. This implies that 
I{Xi-Il{X)\ Mj^CY, = 0) > IC^,25(LPCE„,p,,,)- 

We now use l{Xi\Xi(X)\ \/ -^^Yj = 0) to bound I{Xi;Yl{X)) from below. By definition, we have 
I{Xi]Il{X)) = H{Xi) - H{Xi\Il{X)). Note that H{Xi) = H{X,\ V^y^ Yj = 0), because the coordinates 
of Xi are independent. Let us now upper bound H{Xi\Il(X)). For each transcript vr and each z G {0, 1}, 
letpz^TT = Pr[n(X) = vrj Vj^j = It follows from Lemma|3]that 



H{Xi\U{X)) = E^[H{Xi\U{X)=7T)] 

1 



1 + ^Pr[\/ y,- = z I n(X) = vr] • h(x, I U{X) = TT, V Yj 

2=0 j^i j^i 

1 Pr[\/ Y- = z] / 

.2 = L V / J ^y- 



1 



2=0 jj^i 

1 

l + ^Pr[\/lS- 

2=0 j^i 

1 

l + 5^Pr[\/lS- 

2=0 j^i 



Y,=z 



■ E-rr 



Pz,n 



— ■H(Xi \ n(X) = vr, V Y,- 

Pr n X = vr \ ^\ y I ' V J 



Y.Pz^.-HiXi I n(X) =vr,\/yJ 



i7^^ 



Note that the entropy of Xi, and therefore also any conditional entropy of Xi, is always bounded by 2p log n- 
log n. Hence 

H{Xi\Il{X)) < l + H(Xi\U{X),\/Yj = 0) +Fr[\/Yj = l]-2pn-logn 



< l + Hi^Xi\ U{X), y Yj = 0j+ Apt log 



n. 



Thus we obtain 

I{Xi-Ii{X)) = H{Xi)-H{X,\Ii{X)) 

> H(Xi\\l Yj = o) - if (Xi I n(X), Y y^. = o) - 1 - Apt log 



n 



> l[Xi-Ii{X) I Y y^. = oj - 1 - V lo: 

> IC^,25(LPCE„,p,,) -I- Apt log n. 



n 



□ 



We can finally prove the main lemma of this section. 
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Proof of Lemma \T4\ Let 11 be a private-randomness protocol for ORt(LPCE„ p ,.) that errs with probability 
at most l/(128n). It follows from Lemmas [T6] and [TT] that 

t 

I{X; U{X)) > nXr-, n(X)) > t ■ IC^,i/(64n) (LPCE„,p,,) - Apt" log n - t. 
1=1 

By definition, this quantity bounds also IC^jt i/(i28n)(0'^t('-PCE„ p r))- Therefore, by Lemma [T5] and 
Lemma |6l we get 



i?i/{i28n)(ORt(LPCE„,p,,)) > n (^^^ - O {ptHog 



n) 



Via standard amplification bounds, iZi/(i28n)(ORt('-PCE„p,r)) < i?i/3(0Rt(LPCE„ p ,,)) • O(logn), 
which gives us 

«i/3(0R*(LPCE„,p,.)) = n (^Te^) - O [pf) . □ 

7 Step 3: Reduction to Set Chasing Intersection 

We give a reduction showing that under specific conditions, a protocol for INTERSECT(SC„^p) can be 
used to create a communication protocol for ORt(LPCE„ p j). 

Lemma 18. Let n, p, r, and t be positive integers such that t^Pr^~^ < n/10. If there is a communication 
protocol for INTERSECT(SC„ p) that uses C bits of communication and errs with probability at most 
1/10, then there is a public-randomness communication protocol for ORf (LPCE„ p that uses C + 2p bits 
of communication and errs with probability at most 2/10. 

Proof. Consider an instance of ORt(LPCE„ p ,.)■ There are 2p players, who have t instances of LPCE„ p 
Each instance of LPCE„ p r consists of two instances of PC„^p. Let /j j and gi^j, where 1 < i < p and 
1 < j < t, be the functions that describe these two instances. For each i G [p], player i knows /j j 
and player p + i knows gi^j. If any of the functions /j j or gi^j is r-non-injective, then the solution to the 
problem is 1. The players can check if this is the case in one round of communication with each player 
communicating only one bit. It therefore suffices to show a protocol that solves ORf(EQUAL(PC„^p)), i.e. 
computes 

t 

V {hAhA- ■ ■ •••))= 5i,i(52,i(. . . gp,j{l) ■ ■ .))), 
i=i 

using C bits communication, under the assumption that no /j ,,■ or gi^j is r-non-injective. To this end, we 
show a randomized reduction of this problem to INTERSECT(SC„^p). 

First, using common randomness, the players select random permutations TTij,pij : [ii] — )• [n] for 
1 < i < p and 1 < j < i. Permutations ai^e selected independently, except that ttij = pij for all 
1 < j < t. Furthermore they are generated using public randomness, so they are known to all players. 
(For functions f : A ^ B and g : B ^ C, we write g o f to denote the function from yl to C such that 
(g o /)(x) = g{f{x)) for all x G A.) For all I < j < t, let 

f'p,j = ^P.i ° fpj 9p,j = Pp,j ° apj- 

For all I < i < p — I and I < j < t, let 

fij = ^id ° fij ° ^i+ij and g-j = pi^ o g^j o p7_^^ .. 
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It is easy to see that an instance of ORt(EQUAL(PC„^p)) with f-j and g'^j is equivalent to the original 
instance with fij and gij. The permutations randomly relabel intermediate and final values with final 
values relabeled in the same way on both sides. 

We construct an instance of INTERSECT(SC„^p) by giving the i-th player, I < i < p, a. function 
ft • M ~^ ^(N) ^^^^ th^t for any x G [n], 

f:{x)={fl^{x):l<j<t} 
and by giving the p + i-th player, I < i < p, a function g* : [n] -^V{[n]) such that for any x e [n], 

gtix)={g',j{x):l<j<t}. 
The goal in this instance is to compute 

(/n/lG • • f;m) • • •)) n atm- ■ ■ a^an) ...)))/ 0- 

The instance of INTERSECT(SC„,p) that we have just defined can be seen as stacking mappings from 
different instances of EQUAL(PC„^p) on top of each other. Instead of following a single function f^j or g^ ■ 
for a given instance j, we follow all of them simultaneously, obtaining subsets of [n] instead of just single 
values in [n] . 

Let us show a likely correspondence between the new instance of INTERSECT(SC„^p) and the original 
instance of ORt(EQUAL(PC„,p)). First, if the resuh of solving the instance of ORt(EQUAL(PC„,p)) is 1, 
then clearly, by following the mappings from the instance of EQUAL(PC„^p) resulting in 1, we also obtain 
an element that belongs to the intersection of two resulting sets in INTERSECT(SC„^p). 

Consider the case that the result of solving the instance of ORf(EQUAL(PC„^p)) is 0. We bound the 
probability that the sets appearing in the instance of INTERSECT(SC„^p) intersect. Each element of these 
two sets can be expressed as 

/l,ai(/2,a2(- • • /p,ap(l) • • •)) 

or 

a'lM (fi'2,fe2(---fi'p,bp(i) •••))' 

respectively, where the sequences ai, . . . , Op and bi, . .. , bp describe which of the instances the mapping 
is followed. There are t^^ different pairs of such sequences. What is the probability that we obtain the 
same value for a specific pair of sequences? We want to show that this probability is bounded by rP~^/n. 
If oi = ... = Op = 6i = ... = bp, then we obtain different values, because the oi-th instance in 
ORj(EQUAL(PC„^p)) results in 0. Suppose now that it is not the case that ai = . . . = ap = bi = . . . = bp. 
If ai 7^ bi, then the probability that we obtain the same value is exactly 1/n, because the final values 
are created by two independent random permutations T^i^ai 

and pi^bi- If fli = bi, let k be the lowest 
number greater than 1 such that ak / a-k-i or bk / &fe-i- Since the functions fiai°---° f'k-i ai ^'^'^ 
g'l^^o . . . o g'f^_i are not r^^^-non-injective and are applied to two values randomly distributed by 7^r,ar 
and "^he probability of colUsion is at most rP~^/n. By the linearity of expectation, the expected size of 
the intersection between the two sets in the instance of INTERSECT(SC„^p) is bounded by t'^'^ ■ r'P~^ jn < 
1/10. By Markov's inequality, the probability that the intersection is nonempty is bounded by 1/10, so the 
probability that the reduction fails is bounded by 1/10. Therefore, if we have a communication protocol 
for INTERSECT(SC„^p) that errs with probability at most 1/10, we can use this protocol to obtain a 
public-randomness protocol for ORf(EQUAL(PC„^p)) that cits with probability at most 2/10, provided no 
function in ORf(EQUAL(PC„^p)) is r-non-injective. □ 
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8 Proof of Main Tool (Theorem H) 



We now combine the results of Steps 1,2,3 to conclude our main communication complexity lower bound 
(Theorem |4] from Section |4l). 

Proof of Theorem^ Let t be a positive integer such that t^Pr^'^ < n/10. Due to the result of Newman ll23l . 
we know that every protocol with public randomness can be simulated using private randomness if we allow 
for using additional 0(number-of-players • log(input-size-in-bits)) communication bits and for increasing 
the probability of error by an arbitrarily small constant. By combining this fact with Lemma [141 we find out 
that any public -randomness protocol that errs with probability at most 2/10 has to use at least 

(tn \ / tn \ 

___ j _ o {pt') - 0{2p . log(t . 2p . n • logn)) = Q [-^^^j - O {p{t' + logn)) 

bits of communication. By applying LemmafTSl we get that any communication protocol for INTERSECT(SC„^p) 
that eiTS with probability at most 1/10 has to use at least 

bits of communication. It is easy to check that t = "^'^^^^^^ satisfies the requirement that t^^r^'^ < n/10. 

ll/{2p) 



Additionally for sufficiently large n, 
communication complexity of the problem becomes 



> i . "i^ZiS. SO we can set t = $7 [ "^If^l ). Finally, the 



n 



l+l/(2p) 



provided n>'d{jp) for some function -d. □ 
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